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DETAILED ACTION 

Claim Objections 

1 . Claims 1 5 and 41 are objected to because of the following informalities: 

Claim 15 recites the limitations "constructing a second histogram for each of the 
selected features using the second measurement information; determining a first 
(assumed for examining purpose: second) area encompassed by each of the first 
(assumed for examining purpose: second) histograms; encoding the first (assumed for 
examining purpose: second) areas of the first (assumed for examining purpose: 
second) histograms in metadata elements of a first (assumed for examining purpose: 
second) hypertext markup language (HTML) document; and". 

Claim 41 "means for constructing a second histogram for each of the selected 
features using the second measurement information; means for determining a first 
(assumed for examining purpose: second) area encompassed by each of the first 
(assumed for examining purpose: second) histograms; means for encoding the first 
(assumed for examining purpose: second) areas of the first (assumed for examining 
purpose: second) histograms in metadata elements of a first (assumed for examining 
purpose: second) hypertext markup language (HTML) document; and". 

Applicant in claims 15 and 41 recites the measurement of selected features of 
first object and second object. In these claims, first area under first histograms are 
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associated with first object and further linked to first HTML document and again first 
area under first histograms are associated to second object and further linked to first 
HTML document. These limitations are inconsistent with the limitations: 

1. Claim 15 - "constructing a second histogram for each of the selected features 
using the second measurement information". 

2. Claim 41 - "means for constructing a second histogram for each of the 
selected features using the second measurement information". 

Therefore, it is assumed by the examiner for the examining purposes that a 
second area under second histograms is calculated for second object and are further 
linked to second HTML document. 

Appropriate correction is required. 

Claim Rejections • 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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3. Claims 1, 2, 27, 28, 40 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Abdel-Mottaleb et al., U.S. Patent No. 5,915,038 in view of 
Tsymbalenko et al., Jan. 9, 2001, "Using HTML Metadata to find relevant images on the 
world wide web". 

Claim 1 recites "retrieving a first object, wherein the first object comprises a first 
measurement information encoded in metadata elements of a first hypertext mark up 
language (HTML) document". Abdel-Mattaleb discloses computing an index key 
(metadata) for a query (first) image where as the query (first) image is an image 
identified (retrieved) by the user for use in retrieving other similar images (column 3, 
lines 40-43; column 4, lines 38-43). Abdel-Mettaleb further discloses the extraction of 
index keys is the extraction of metadata, where extracting metadata is the measurement 
of the features of the still images (column 7, lines 17-23). 

Claim 1 further recites "comparing the first object with a second object, wherein 
the second object comprises a second measurement information encoded in metadata 
elements of a second hypertext markup language (HTML) document". Abdel-Mettaleb 
discloses the comparison of the index key (metadata information) of the query (first) 
image with each index key of the corresponding image (second) being searched 
(column 3, lines 45-50). 

Claim 1 recites "retrieving 'the second object in response to the difference 
between the first measurement information of the first HTML document and the second 
measurement information of the second HTML document being less than or equal to a 
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threshold difference value". Abdel-Mettaleb discloses the selection of the second image 
with respect to the threshold difference value (column 3, lines 50-60). 

Abdel-Mettaleb further discloses that the still images may be stored locally or 
remotely, such as a remote Web site on the Internet and archived using the index key 
extraction and archival process (column 8, lines 13-16). Abdel-Mettaleb does not 
explicitly teach the encoding of metadata information in metadata elements of a HTML 
document. 

However, Tsymbalenko discloses the image retrieval system where the images 
are accessed through HTML documents and that the bulk of the content of HTML 
documents is textual whereas the textual content and the structure of HTML documents 
are considered to be "metadata" describing the images and this metadata is used to 
determine which images may be relevant to a query (page 3, paragraph 2). 
Tsymbalenko further discloses the search strategy where the result showed a HTML 
document with links to other HTML documents that best matched the query (page 3, 
paragraph 5) and every image is linked to each HTML document (page 3, para. 6, lines 
2-3). 

Therefore, it would had been obvious to one having ordinary skill in the art at the 
time of the invention was made to include the teachings by Tsymbalenko in the 
invention of Abdel-Mettaleb. One would have been motivated to use the concept of 
image metadata encoding in the HTML documents by Tsymbalenko in the invention of 
Abdel-Mettaleb because both the references are directed to image search and retrieval. 
Abdel-Mettaleb discloses that the still images may be stored locally or remotely, such as 
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a remote web site on the internet and Tsymbalenko further provides the image search 
and retrieval using the HTML documents where each image index key (metadata) 
extracted from Abdel-Mettaleb's invention can be encoded in HTML document as a 
textual information and the advantage of using metadata textual information would lead 
in saving memory and bandwidth on the image retrieval system. 

Claim 2 has been analyzed and rejected as per claim 1. 

Claim 27 additionally recites the system that retrieves images by content 
measure metadata encoding. Abdel-Mettaleb discloses the use of computer system for 
his invention (column 6, lines 24-37). All other limitations in claim 27 had been analyzed 
and rejected as per claim 1 . 

Claim 28 has been analyzed and rejected as per claims 1 and 27. 

Claim 40 has been analyzed and rejected as per claim 27. 

4. Claims 3, 9-12, 14, 29, 35-38 and 40 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Eakins et al., Jan. 1999, "Content-based image retrieval: a 
report to the JISC technology applications programme", in view of Opittek et al., U.S. 
Patent No. 3,979,555, and further in view of Tsymbalenko et al., Jan. 9, 2001, "Using 
HTML Metadata to find relevant images on the world wide web". 
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Claim 3 recites "A method of encoding images by content measure metadata 
encoding, comprising: measuring selected features of an object to form measurement 
information; constructing a histogram for each of the selected features using the 
measurement information". Eakins discloses content-based image retrieval (CBIR) 
methods (page 18) and systems (page 23) where stored images are retrieved from a 
collection by comparing features automatically extracted from the images themselves 
where the commonest features used are mathematical measures of color, texture or 
shape (page 18, para. 3). Eakins further discloses measuring the proportion of pixels of 
each color within the image and a color histogram is computed to show this 
measurement (page 18, para. 4, lines 2-4). 

Claim 3 further recites "determining an area encompassed by each of the 
histograms". Eakins does not teach of calculating an area encompassed by histogram 
for each of the selected features. However, Opittek discloses calculating a total area 
under histogram to compute the total distribution of gray scale intensity levels in the 
image (column 3, lines 13-20). Opittek discloses a histogram in figure 1-4 where x-axis 
represents internal division of the selected feature (intensity of a color) and y-axis 
represents a frequency of occurrence of intensity level. 

Therefore, it would had been obvious to one having ordinary skill in the art at the 
time of the invention was made to use the concept of calculating area under the 
histogram by Opittek in the invention of Eakins. One would have been motivated to use 
the concept of calculating area under the histogram by Opittek in the invention of Eakins 
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because Eakins computes a color histogram for each color within the image and 
Opittek's further shows the histogram in standard x-y coordinate format with x-axis 
representing internal division of the selected feature (intensity of a color) and y-axis 
representing a frequency of occurrence of intensity level in figure 1 and then calculates 
a total area under this histogram to compute the total distribution of color intensity 
levels in the image and calculating area will provide a mathematical value of the 
distribution of the color intensity levels. 

Claim 3 further recites "encoding areas of the histograms in metadata elements 
of a hypertext markup language (HTML) document". Eakins discloses the encoding of 
parameters such as the color space on which the histogram is based in metadata 
elements of a XML (Extensible Markup Language) document. Eakins does not teach 
encoding in metadata elements of a HTML document. Encoding in metadata of HTML is 
well known in the art and this is supported by Tsymbalenko et al., which further 
discloses that each image is associated to each HTML document as explained n claim 
1. 

Claim 9 recites "A method of claim 4, wherein measuring selected features 
further comprises measuring a geometric feature of the object". The features such as 
shape, size, distance, rotation angle, etc., are standard types of geometric features 
depending on the object type Therefore claim 9 has been analyzed and rejected as per 
claim 3. 



Application/Control Number: 10/087,347 Page 9 

Art Unit: 2625 

Claim 10 and 11 had been analyzed and rejected as per claim 9. 

Claim 12 has been analyzed and rejected as per claim 3. 

Claim 14 has been analyzed and rejected as per claim 3 and in further in view of 
claim 1 . 

Claim 29 recites a system that performs the method as recited in claim 3. Eakins 
further discloses many content-based image retrieval systems and these systems use 
the same method as recited in claim 3 (page 23; page 18, para. 3, lines 3-4). Therefore, 
claim 29 has been analyzed and rejected as per claim 3. 

Claim 35 has been analyzed and rejected as per claim 29 and further in view of 
claims 3 and 9. 

Claims 36 and 37 had been analyzed and rejected as per claim 35. 

Claim 38 has been analyzed and rejected as per claim 29 and further in view of 
claim 3 and 12 

Claim 40 has been analyzed and rejected as per claim 29 and in further view of 
claims 3,14 and 1. 
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5. Claims 4-8 and 30-34 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Eakins et al., Jan. 1999, "Content-based image retrieval: a report to the JISC 
technology applications programme", in view of Opittek et al., U.S. Patent No. 
3,979,555, and further in view of Tsymbalenko et al., Jan. 9, 2001, "Using HTML 
Metadata to find relevant images on the world wide web", and further in view of Baxes, 
1994, Book Publication, "Digital image processing: principles and applications". 

Claim 4 recites "A method of claim 3, wherein measuring selected features 
further comprises measuring an intensity of a preselected color of the object". As 
explained in rejection of claim 3, the combined invention of Eakins and Opittek 
determines intensity levels of each color based on the histograms. Therefore claim 4 
has been analyzed and rejected as per claim 3. 

It is well known in the art that each color can be red, green or blue as in RGB 
space and each color can also be a gray color in a gray-scale image. This well known 
art is further support by Baxes, who in his book printed in 1994, discloses that three 
histograms can be generated in an RGB space for each color component (page 63, 
para 3; page 64, para 1 and figures 3.20(a), 3.20(b) and 3.20(c). Therefore, it would had 
been obvious to one having ordinary skill in the art at the time of the invention was 
made to use the concept of constructing histograms for a preselected color as disclosed 
by Baxes. One would have been motivated to use the concept of Baxes because each 
histogram can help to determine the brightness distributions, contrast, and dynamic 
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ranges of the individual components where brightness is the measured intensity of the 
pixels in the image (page 3). 

Claims 5, 6, 7, and 8 had been analyzed and rejected as per claim 4. 

Claims 30, 31, 32, 33, 34 had been analyzed and rejected as per claim 29 and in 
further view of claims 4, 5, 6, 7 and 8. 

6. ' Claims 13 and 39 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Eakins et al., Jan. 1999, "Content-based image retrieval: a report to the JISC 
technology applications programme", in view of Opittek et al, U.S. Patent No. 
3,979,555, and further in view of Tsymbalenko et al., Jan. 9, 2001, "Using HTML 
Metadata to find relevant images on the world wide web" and further in view of Rorvig, 
"An experimental approach for the content-based image analysis: an open source 
agenda for research". 

Claim 13 recites "a method of claim 3, further comprising converting the area 
under the histogram to a Lorenz Information Measure (LIM)'\ Eakins, Opittek and 
Tsymbalenko do not teach converting the area under the histogram to a Lorenz 
Information Measure (LIM). However, Rorvig teaches converting the area under the 
histogram to a Lorenz Information Measure (page 14 and 15). Using LIM to convert the 
area under the histogram is well known in the field of distributional analysis, such as 
economics. 
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Therefore, it would had been obvious to one having ordinary skill in the art at the 
time of the invention was made to use the invention of Rorvig in the combined invention 
of Eakins, Opittek and Tsymbalenko. One would have been motivated to use Rorvig's 
invention in the combined invention of Eakins, Opittek and Tsymbalenko because using 
Rorvig's concept of using LIM on the area of compassed by the histogram will convert 
histogram values to a single value and then this single value can be encoded as 
metadata name tags in HTML as per Tsymbalenko rather than attempting to use all of 
the individual data in the histogram. 

Claim 39 has been analyzed and rejected as per claim 29 and in further view of 

13. 

7. Claims 15, 21-24, 26, 41, 47-50 and 52 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Abdel-Mottaleb et al., U.S. Patent No. 5,915,038, in view of 
Eakins et al., Jan. 1999, "Content-based image retrieval: a report to the JISC 
technology applications programme", and further in view of Opittek et al., U.S. Patent 
No. 3,979,555, and further in view of Tsymbalenko et al., Jan. 9, 2001, "Using HTML 
Metadata to find relevant images on the world wide web". 

Claim 15 recites a method of retrieving images by content measure metadata 
encoding. As explained in the rejection of claim 1 , Abdel-Mottaleb, extract (measures) 
an index key (metadata) for the query (first) image and then generates the key index for 
each of the images to be searched, where extracting metadata is the measurement of 
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the features of the still images, and further discloses the selection of the second image 
with respect to the threshold difference value. Abdel-Mettaleb does not go into the 
details of constructing histogram for each feature measured, then generating area under 
histogram and further encoding the areas under histogram in metadata elements of a 
HTML document. 

However, the invention of Eakins combined with Opittek and Tsymbalenko, as 
explained in the rejection of claim 3, provides the detailed steps of the image analysis 
required on the query (first) image and all other images to be searched and further 
provides the concept of encoding of metadata in metadata elements of a HTML 
document. Therefore, it would had been obvious to one having ordinary skill in the art at 
the time of the invention was made to use the combined invention of Eakins, Opittek 
and Tsymbalenko in the invention of Abdel-Mettaleb. One would have been motivated 
to use the combined invention of Eakins, Opittek and Tsymbalenko in the invention of 
Abdel-Mettaleb because Abdel-Mettaleb, Eakins, and Tsymbalenko are directed to the 
methods and systems of image search and retrieval and Eakins further go more in detail 
of using content features in the image search and retrieval to optimize the search 
techniques and have better image similarity results and Opittek further provides the 
concept of image analysis on the selected image features such as obtaining a total 
mathematical measure (metadata) of color intensity distribution and Tsymbalenko 
further supports Abdel-Mettaleb method of searching the images on web by encoding 
the metadata of the images in the metadata elements of the HTML document as 
explained in the rejection of claim 1 . 
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Claim 21 has been analyzed and rejected as per claim 15 and further in view of 
claims 3 and 9. 

Claim 22 and 23 had been analyzed and rejected as per claim 21. 

Claim 24 has been analyzed and rejected as per claim 15 and further in view of 
claims 3 and 14. 

Claim 26 has been analyzed and rejected as per claim 15 and further in view of 
claim 14, 3 and 1. 

Claim 41 recites a system that performs the method recited in claim 15. Claim 41 
has been analyzed and rejected as per claim 27. 

Claim 47 has been analyzed and rejected as per claim 41 and in further view of 
claim 21. 

Claims 48 and 49 has been analyzed and rejected as per claim 47. 

Claim 50 has been analyzed and rejected as per claim 41 and further in view of 
claim 24. 
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Claim 52 has been analyzed and rejected as per claim 41 and further in view of 
claim 26. 

8. Claims 16-20 and 42-46 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Abdel-Mottaleb et al., U.S. Patent No. 5,915,038, in view of Eakins et 
al., Jan. 1999, "Content-based image retrieval: a report to the JISC technology 
applications programme", and further in view of Opittek et al., U.S. Patent No. 
3,979,555, and further in view of Tsymbalenko et al., Jan. 9, 2001, "Using HTML 
Metadata to find relevant images on the world wide web", and further in view of Baxes, 
1994, Book Publication, "Digital image processing: principles and applications". 

Claim 16 recites "A method of claim 15, wherein measuring selected features 
further comprises measuring an intensity of a preselected color of the object". Abdel- 
Mottaleb does not teach of measuring intensity of a preselected color of the object but 
measuring of intensity of color with motivation has been explained in the rejection of 
claim. The claim 16 has been analyzed and rejected as per claim 15 and in further view 
of claim 4. 

Claims 17-20 had been analyzed and rejected as per claims 16 and 15 and 
further in view of claims 5-8. 



Application/Control Number: 10/087,347 Page 16 

Art Unit: 2625 

Claims 42-46 had been analyzed and rejected as per claims as per claim 41 and 
in further view of claims 16-20. 

9. Claims 25 and 51 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Abdel-Mottaleb et al., U.S. Patent No. 5,915,038, in view of Eakins et al., Jan. 
1999, "Content-based image retrieval: a report to the JISC technology applications 
programme", and further in view of Opittek et al., U.S. Patent No. 3,979,555, and further 
in view of Tsymbalenko et al., Jan. 9, 2001, "Using HTML Metadata to find relevant 
images on the world wide web" and further in view of Rorvig, "An experimental 
approach for the content-based image analysis: an open source agenda for research". 

Claim 25 recites "a method of claim 15, further comprising converting the area 
under the histogram to a Lorenz Information Measure (LIM)". Abdel-Mottaleb, Eakins, 
Opittek and Tsymbalenko do not teach converting the area under the histogram to a 
Lorenz Information Measure (LIM). However, Rorvig teaches converting the area under 
the histogram to a Lorenz Information Measure (page 14 and 15) as explained in the 
claim 13 with motivation. Therefore, claim 25 has been analyzed and rejected as per 
claim 25 and in further view of claim 1 3. 

Claim 51 has been analyzed and rejected as per claim 25. 



Conclusion 
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10. The prior art of record and not relied upon is considered pertinent to applicant's 
disclosure: 

• Kim et al., U.S. Patent No. 6,754,667, discloses a content based image retrieval 
system and method of retrieving image using the same. 

• Tsujimura et al., U.S. Patent No. 5,586,197, discloses image searching method 
and apparatus thereof using color information of an input image. 

• Mukherjea et al., U.S. Patent No. 6,415,282, discloses a method and apparatus 
for query refinement. 

• Yoon et al., U.S. Patent No. 6,621,926, discloses a image retrieval system and 
method using image histogram. 

• Barber et al., U.S. Patent No. 5,579,471 , discloses a image content based image 
query system and method. 

• Golshani et al., U.S. Patent No. 6,594,386, discloses a method for computerized 
indexing and retrieval of digital images based on spatial color distribution. 

• Murakawa, U.S. Patent No. 6,463,432, discloses an apparatus for and method 
for retrieving images. 

• Ito et al., U.S. Patent No. 5,555,318, discloses a thresholding method for 
segmenting gray scale image, method for determining background concentration 
distribution, and image displacement detection method and also disclose the 
concept of area under the histogram. 
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• Stapleton et al., U.S. Patent No. 5,832,140, discloses automated quality 
assurance image processing system and further provides the concept of area 
under histogram. 

• Wang, U.S. Patent No. 6,373,979, discloses a system and method for 
determining a level of similarity among more than one image and a segmented 
data structure for enabling such determination. 

• Shimura et al., U.S. Patent No. 5,644,765, discloses a image retrieving method 
ad apparatus that calculates characteristic amounts of data correlated with and 
identifying an image. 

• Jain et al., U.S. Patent No. 5,893,095, discloses a similarity engine for content- 
based retrieval of images. 

• Jain et al., U.S. Patent No. 5,915,250, discloses a threshold based comparison 
system and method for image search and retrieval. 

• Tang et al., IEEE Publication, 2000, "A content-based image retrieval system on 
the mode of network", (pp. 422-425) 

• Pearce et al., IEEE Publication, 1994, "Theoretical and experimental comparison 
of the Lorenz Information Measure, Entropy, and the Mean Absolute Error", (pp. 
24-29) 

• Pass et al., IEEE Publication. 1996, "Histogram refinement for content-based 
image retrieval", (pp. 96-102) 

• Yuwono et al., IEEE Publication, 1996, "Search and ranking algorithms for 
locating resources on the world wide web", (pp. 164-171) 
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• Lee et al., IEEE Publication, 1994, "Query by image content using multiple 
objects and multiple features: user interface issues", (pp. 76-80) 

• Pala et al., IEEE Publication, 2000, "Using multiple examples for content-based 
image retrieval", (pp. 335 -338). 

• Newsome et al., IEEE Publication, 1997, "HperSQL: Web-based query interface 
for biological databases", (pp. 329-339). 

• Chen et al., IEEE Publication, 1999, "A synchronized and retrieval video/HTML 
lecture system for industry employee training", (pp. 750-755). 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Manav Seth whose telephone number is (703) 306- 
4117. The examiner can normally be reached on Monday to Friday from 8:30 am to 
5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, examiner's trainer, 
Bhavesh Mehta, can be reached on (703) 305-3885. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published 
applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For 
more information about the PAIR system, see http://pair-direct.uspto.gov. Should you 
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have questions on access to the Private PAIR system, contact the Electronic Business 
Center (EBC) at 866-217-9197 (toll-free). 
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